Non-Model-Based Algorithm Portfolios for SAT

When tackling a computationally challenging combinatorial problem, one often observes that some solution approaches work well on some instances, while other approaches work better on other instances. This observation has given rise to the idea of building algorithm portfolios [5]. Leyton-Brown et al. [1], for instance, proposed to select one of the algorithms in the portfolio based on some features of the instance to be solved. This approach has been blessed with tremendous success in the past. Especially in SAT, the SATzilla portfolios [7] have performed extremely well in past SAT Competitions [6]. We investigate alternate ways of building algorithm portfolios that differ substantially from the way SATzilla assembles a portfolio. The key idea behind SATzilla is to train a runtime prediction model for each constituent solver, based on a number of well-engineered features of SAT instances. Given a new instance, SATzilla predicts the runtime of each candidate solver based on instance features and the trained models, and chooses the solver that is predicted to perform the best. In contrast, we consider non-model-based machine learning techniques such as simple k-nearest-neighbor (k-NN) classification to determine which solver to use to tackle a given instance. Our motivation stems from two observations: (a) accurately predicting the runtime of sophisticated SAT solvers is a very challenging task; indeed, the runtime predictor underlying SATzilla can be even orders of magnitude off from the true runtime; and (b) while fast and accurate runtime prediction is certainly sufficient for building a solver portfolio, it is by no means necessary. In fact, it would suffice entirely if we could predict the fastest solver without having any knowledge of how long it will actually take to solve the given instance. This idea has found success in fields adjoining SAT, for example in portfolios for the quantified Boolean formula (QBF) problem [4], for general constraint satisfaction problems (CSPs) [3], and to some extent even for SAT itself [2]. Our portfolio works as follows. In the learning phase, we are given a pool T of training instances, a function that provides features for any given problem instance (we use the 48 core SATzilla features here), a set S of constituent solvers forming the portfolio, and a timeout t. We compute the runtime (with cutoff t) for all solvers on all instances as well as normalization parameters so that all features for all instances in the training set populate the interval [0, 1]. At runtime, given a new instance I, we compute its features, normalize them, and compute the set TI ⊂ T consisting of k training instances closest to I in terms of Euclidean distance. Then, for each solver S ∈ S, we compute the